ProgrammingThis forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.
Notices
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
So, you'll be glad to know the the post I've marked as resolved. I have a solution which works. I think I was trying to get perfect data back from the remote server. I've searched and searched and maybe Python just isn't as clever as I thought it was. Who knows..
Anyway, I'm just returning all data (which in the real world will be a lot!):
Code:
#!/usr/bin/python
#
from subprocess import *
ssh = Popen(["ssh", "root@boot", 'xtprocadmin'],
shell=False,
stdout=PIPE,
stderr=PIPE)
result = ssh.stdout.readlines()
lines = result.splitlines()
for line in lines[1:]:
columns = line.split()
status = columns[3]
nodes = columns[2]
if status == "down":
print "%s is down" % nodes
So, I've tested the above with some tests, so I'm pretty sure that the theory is solid (ish) - But could you help with my loop?
I'm trying to take the data returned from the ssh session, and break it into lines.. then into columns. Then find the columns that contain the word down.. then print (for now) the node which is down.
I thought the above would work, but clearly I'm missing something simple.
I've emboldened the code which I think you're misunderstanding.
Code:
#!/usr/bin/python
#
from subprocess import *
ssh = Popen(["ssh", "root@boot", 'xtprocadmin'],
shell=False,
stdout=PIPE,
stderr=PIPE)
result = ssh.stdout.readlines()
lines = result.splitlines()
for line in lines[1:]:
columns = line.split()
status = columns[3]
nodes = columns[2]
if status == "down":
print "%s is down" % nodes
Last edited by Myk267; 02-10-2017 at 09:14 PM.
Reason: Silly me, I don't need to quote the entire post...
I've emboldened the code which I think you're misunderstanding.
Code:
#!/usr/bin/python
#
from subprocess import *
ssh = Popen(["ssh", "root@boot", 'xtprocadmin'],
shell=False,
stdout=PIPE,
stderr=PIPE)
result = ssh.stdout.readlines()
lines = result.splitlines()
for line in lines[1:]:
columns = line.split()
status = columns[3]
nodes = columns[2]
if status == "down":
print "%s is down" % nodes
Hi, and many thanks for responding.
My understanding of it is (which is likely wrong, as I'm learning):
Loop through the lines of the data and separate the words in that line into columns. But I added [1:] as I don't want the headings. I will try to emphasize my understanding of what I've written.
lines = result.splitlines()
for line in result[1:]:
I see that I was trying to split the lines in the output and then loop through them and split them again into columns.
I changed it for:
Code:
from subprocess import *
ssh = Popen(["ssh", "root@boot", 'xtprocadmin'],
shell=False,
stdout=PIPE,
stderr=PIPE)
result = ssh.stdout.readlines()
for line in result[1:]:
columns = line.split()
status = columns[3]
nodes = columns[2]
if status == "down":
print "%s is down" % nodes
This seems to work actually, but I need to do some more testing to make sure it's solid.
Thank you for highlighting the area which needed attention.
I wouldn't say that I FULLY understand what I've done, it's a bit of a fudge.. and I'm pretty sure it's not the best way of doing it.
I thought the above would work, but clearly I'm missing something simple.
what kind of problem did you experience?
I would try to add some debug line to the code to see what's happening, later you can remove them:
Code:
for line in lines[1:]:
print "line=" + line
columns = line.split()
status = columns[3]
print "status=" + status
nodes = columns[2]
print "nodes=" + nodes
if status == "down":
print "%s is down" % nodes
but obviously you can modify it as you wish to make it really usable (=that means it will give you information about what's going on)
My understanding of it is (which is likely wrong, as I'm learning):
Loop through the lines of the data and separate the words in that line into columns. But I added [1:] as I don't want the headings. I will try to emphasize my understanding of what I've written.
Thanks
Jon
In that case, we both get to be wrong. Fantastic.
See, I had figured that what you really wanted was to drop the first column (just my guess), which was throwing your column indexing off, but now we know that you didn't want the header. So now all you've got to do is fix up your error in indexing into the columns list. Python lists start counting from 0.
I would try to add some debug line to the code to see what's happening, later you can remove them:
Code:
for line in lines[1:]:
print "line=" + line
columns = line.split()
status = columns[3]
print "status=" + status
nodes = columns[2]
print "nodes=" + nodes
if status == "down":
print "%s is down" % nodes
but obviously you can modify it as you wish to make it really usable (=that means it will give you information about what's going on)
Great advice regarding extra steps for debugging. Not only will that help me to debug the code, even in a working example it will help me to fully understand what I'm doing I'll remember to do that more
It works now, kind of as expected. But there are a couple of other little things which aren't quite right. But I think it's going to be tough to get exactly what I want. This particular example works okay, but that's because my VM Test environment is a fudge. 'xtprocadmin' is a custom tool that we use which responds with the status of each node in the cluster. For development purposes I have written a bash one liner to cat out a file which has 15 or so lines of an xtprocadmin output. In the real world, when I use this script.. xtprocadmin will respond with nearly 7000 nodes. Which is in turn a lot more data for it to manage. But having said that, thinking while I'm typing; 7000 lines really is not a lot for python to deal with, nor is it really too much for ssh to deal with since it's plain text. So I'm likely worrying over nothing.
See, I had figured that what you really wanted was to drop the first column (just my guess), which was throwing your column indexing off, but now we know that you didn't want the header. So now all you've got to do is fix up your error in indexing into the columns list. Python lists start counting from 0.
Have a nice day
I don't think I need to fix the column indexing, as it works as I'd expect it to (in a sense). I was aware of the column indexing being 0 relative - I did a Python course online yesterday lol.
This is my code so far, I have put some error checking in for the ssh session, and added some more to the loop for displaying another node status type.
Code:
#!/usr/bin/python
#
from subprocess import *
import sys
ssh = Popen(["ssh", "root@boot", 'xtprocadmin'],
shell=False,
stdout=PIPE,
stderr=PIPE)
result = ssh.stdout.readlines()
if result == []:
error = ssh.stderr.readlines()
print >>sys.stderr, "ERROR: %s" % error
else:
for line in result[1:]:
columns = line.split()
status = columns[3]
nodes = columns[2]
if status == "down":
print "%s is down" % nodes
if status == "admindown":
print "%s is admindown" % nodes
The only gripe I have so far is with ordering the output. At the moment the output is as such:
Code:
[root@jons-server ~]# ./test.py
c2-0c2s4n1 is down
c2-0c1s4n2 is down
c2-0c1s15n0 is down
c2-0c1s8n3 is admindown
c2-0c2s2n1 is down
c2-0c1s6n1 is admindown
c2-0c1s15n3 is down
c2-0c1s1n3 is down
c2-0c1s6n1 is admindown
I don't like how it mixes down and admindown. I guess it's being displayed in the order it is in the test file. However it would be nice to order it by columns[3] - I'm sure it's possibly and probably straight forward. However it's not critical, more of a nice to know... as the plans for this would be to now put the node names into another process/function and search log files using the node name as a search criteria and then filter results according to keywords to look out for, like CATERR or MCE or CORRECTABLE etc.
In the course I did online there was a section on opening files, and reading them line by line.. so hopefully that will get me started on this.
This is not something that needs to be written at all, I have written shell scripts which do all of this.. but I want to learn Python and thought what better way than to give myself a challenge.
Last edited by jonnybinthemix; 02-11-2017 at 05:10 AM.
there are sort and filter functions available in python, so theoretically you can do something like this:
Code:
for line in filtered result[1:]:
- or
for line in sorted result[1:]:
but it is not real python code at this moment, you need to specify how to sort/filter. If you really wish to go into details just tell us....
I initially just wanted to have all the "down" together and all the "admindown" together. It's not all that important though.. it's more just for my understanding
I initially just wanted to have all the "down" together and all the "admindown" together. It's not all that important though.. it's more just for my understanding
This it not really relevant.. as I don't really want to display the down nodes at all yet in the script.. it was merely for my own knowledge.
I have been working on what to do with the list of nodes after I have them, and that would be to collate them into a list and/or variable or something, to be able to loop through them and check for their existence in a set of log files.
I have played with the following:
Code:
#!/usr/bin/python
#
from subprocess import *
import sys
ssh = Popen(["ssh", "root@boot", 'xtprocadmin'],
shell=False,
stdout=PIPE,
stderr=PIPE)
result = ssh.stdout.readlines()
if result == []:
error = ssh.stderr.readlines()
print >>sys.stderr, "ERROR: %s" % error
else:
for line in result[1:]:
columns = line.split()
status = columns[3]
nodes = columns[2]
if status == "down":
#print "%s is down" % nodes
down_nodes = nodes
if status == "admindown":
#print "%s is admindown" % nodes
admin_down_nodes = nodes
print down_nodes+admin_down_nodes
#down = down_nodes+admin_down_nodes
However, when I test the two variables down_nodes and admin_down_nodes I just see the first line. I guess this is because it's overwriting the variables on each iteration of the loop, so in actual fact I'm probably seeing the last line? You can see I experimented with the ability to take those two other variables and create a single variable with all node names... however, is there a way that I can tidy this up by creating a variable in which each iteration of the loop for both instances (down and admindown) will be appended to the variable, and as such I will end up with one variable with a simple list of node names?
if status == "down":
#print "%s is down" % nodes
down_nodes = down_nodes.append(nodes)
if status == "admindown":
#print "%s is admindown" % nodes
down_nodes = down_nodes.append(nodes)
It feels like I'm starting to think the right way, but still new enough to mess up all the time. Am I close?
EDIT:
The error I was getting with the above was "down_nodes not defined" so I added down_nodes = "" to the top, in a vein attempt to declare it as a variable before the loop runs. Which did change the error message to: AttributeError: 'str' object has no attribute 'append'
EDIT AGAIN:
I think I have just worked out myself that append() is just for lists, and not for strings. Hmmm... thinking......
Last edited by jonnybinthemix; 02-11-2017 at 02:18 PM.
#!/usr/bin/python
#
from subprocess import *
import sys
ssh = Popen(["ssh", "root@boot", 'xtprocadmin'],
shell=False,
stdout=PIPE,
stderr=PIPE)
result = ssh.stdout.readlines()
down_nodes=[]
if result == []:
error = ssh.stderr.readlines()
print >>sys.stderr, "ERROR: %s" % error
else:
for line in result[1:]:
columns = line.split()
status = columns[3]
nodes = columns[2]
if status == "down":
down_nodes = down_nodes.append(nodes)
if status == "admindown":
down_nodes = down_nodes.append(nodes)
print down_nodes
I thought with down_nodes=[] I would be declaring it as a list (albeit empty) - and then with down_nodes = down_nodes.append(nodes) I would be appending the list. But maybe.............
....as I was typing, I realised.. I'm thinking too much like a shell script!! var = var+something!
I changed it to:
Code:
if status == "down":
down_nodes.append(nodes)
if status == "admindown":
down_nodes.append(nodes)
So now it works, when I print it out.. it's not very attractive, but it doesn't need to be.
Sorry, I probably don't need to post any of this, but I figure if someone else embarks on the same or similar journey as I am, my thoughts along the way may be helpful to them.
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.